Significance levels for multiple tests

Let X 1 . . . . . X, be n random variables, with cumulative distribution functions F1, . . . , F,. Define ¢i := Fi(Xi) for all i, and let ~(1) ~< ... ~< ~(,) be the order statistics of the (~)~. Let cq ~< ... ~< ~. be n numbers in the interval [0, 1]. We show that the probability of the event R := {(") ~< ct~ for all 1 ~< i ~< n} is at most min~n~Ji }. Moreover, this bound is exact: for any given n marginal distributions (F~)~, there exists a joint distribution with these marginals such that the probability of R is exactly min~nct~/i}. This result is used in analyzing the significance level of multiple hypotheses testing. In particular, it implies that the ROger tests dominate all tests with rejection regions of type R as above. © 1997 Elsevier Science B.V.